The role of peer-reviewers in checking supporting information

Laurent Gatto

The role of peer-reviewers in checking supporting information promoting open science

Laurent Gatto                      Computational Proteomics Unit
https://lgatto.github.io           University of Cambridge
lg390@cam.ac.uk                    @lgatt0

Slides: http://bit.ly/20170329peerrev

(Last update Wed Mar 29 22:11:47 2017)

About me

My responsibility as a reviewer

Hence

(I don’t see novely, relevance, news-worthyness as my business as a reviewer. I leave that to the editor.)

Quick survey

The above implies that

Review can be very quick

If no data/software are available, there can’t be any review.

Hence open science and reproducible research are essential: are data/software/method accessible, understandble, reproducible?

How much should we invest? (1)

How much should we invest? (2)

Tips (1): availability

Timing: 5 - 10 minutes

Constructive comments

Tips (2): do numbers match?

Tips (3): meta-data

No need to necessarily repeat everything.

But if I wanted to repeat the analysis, would I be able to? Is something missing? What?

Tip: Document everything in a README file.

Tips (4): what data, what format

(Remember tip 2 - do numbers match!)

Tip (5): license

Open science is not only about accessing outputs for free (as in beer), but for free (as in speech), i.e. in a way that allows to re-use.

If you share anything, make sure users are allowed to re-use it, and are aware of the terms under which they can re-use it.

NB: Ask you default data repository about licensing. You might be surprised!

Quick survey

Data/software citation

As somebody who values data, software and methods, I will make every effort to make sure that these are cited when reviewing papers.

How to cite software/data

Where to share data?

Where to expect to find it? From best to least desirable:

There is no perfect solution, often a combination of the above is great.

What matters

FAIR principles:

Why not SI: not FAIR, not discoverable, not structured, voluntary, SI used to bury stuff.

My ideal review system (1)

From Chris Hartgenink, 14 March 2017, OpenConCam

Biais in peer review:

2 stage review: start with intro and methods, then results.

My ideal review system (2)

  1. Submit you data to a repository, where it get’s checked (by specialists, data scientists, data curators) for quality, annotation, meta-data.
  2. Submit you research with a link to the peer reviewed data.

Automation

Thank you

The content of this repository is made available under the Creative Commons Attribution license.